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A variety of interventions have been attempted in education and other fields to 
increase the use of research use in policy and practice. However, there is still 
limited research on the impact of these interventions. This paper uses survey and 
qualitative data to analyze three interventions designed to increase research use 
among secondary school leaders in nine Canadian school districts. These 
interventions were found to have little impact, but were more successful where (1) 
designated facilitators were involved and (2) research used was connected to 
existing priority issues. The research design for this study (measuring the change 
in agreement with particular bodies of research knowledge using interventions and 
pre-post design) is a promising methodology to measure both research use and 
impact. 


Introduction 

This article reports on efforts to increase education leaders’ knowledge of empirical 
evidence related to secondary schooling through three modest interventions in participating 
school districts. The intent of this work is to improve our understanding of the current role and 
impact of research in Canadian schools, and also to examine ways in which the awareness and 
use of relevant research could be strengthened. The study is part of a broader program (Research 
Supporting Practice in Education - www.oise.utoronto.ca/rsep) of understanding the ways in 
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which research can inform better policy and practice, which our team calls Knowledge 
Mobilization (KM). 

This study, funded by a national educational organization, involved the collaboration of a 
university research team and eleven school districts in four provinces across Canada from 2007 
to 2009. The study was focused on superintendents, principals and others with designated 
leadership roles in secondary schools or districts. The overall study explored district leaders’ 
knowledge of relevant research on improving secondary education, the main sources of that 
knowledge, the practices districts used to share research and the effects of some interventions on 
those practices and knowledge (Levin, Sa, Cooper, & Mascarenhas, 2009). One element of the 
study, explored in this paper, was to assess the impact of three fairly simple interventions, 
derived from the literature, on research knowledge and use. Simple interventions were chosen as 
potentially feasible taking into account the time and other pressures on practitioners. 

Related Literature and Conceptual Framework 

There is growing momentum internationally around making more use of research 
knowledge to shape policy and practice, not only in education but in most spheres of public 
policy. This interest is manifested not only in terms of rhetorical commitment by governments 
and increased media interest, but also by additional resources and growing capacity in various 
organizations (Cooper, Levin, & Campbell, 2009; Levin, 2011; OECD, 2007). Efforts to 
understand and improve the relationship between research and practice are more frequent in 
health (Amo & Cousins, 2007; Dobbins et al., 2007; Lavis, Lomas, Hamid, & Sewankambo, 
2006) than they are in education, although there are some interesting instances in education as 
well (e.g. Coburn, Honig, & Stein, 2010; Cordingley, 2008; Rickinson, 2005; Sebba, 2007). 
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There are many different terms and definitions in the literature related to the connections 
between research and practice. For purposes of this research, we define knowledge mobili z ation 
as attempts to integrate research evidence into policy and practice. KM research, then, is about 
understanding how research makes its way into organizations in ways that result in changes in 
ideas, policies and practices. 

Both the terms research and use have been defined in many different ways. For the 
purposes of this study, research is defined to mean the systematic gathering and use of data or 
other forms of empirical evidence to address a theoretical, practical, or policy problem. Along 
with most scholars (e.g. Nutley, Walter, & Davies, 2007; Weiss, 1979) we recognize that the use 
of research can take a variety of forms and can occur over varying periods of time. 

A growing body of empirical and conceptual work, much of it in fields other than 
education (e.g. Belkhodja, Amara, Landry, & Ouimet, 2007; Lavis, 2006; Lemieux-Charles & 
Champagne, 2004; Mitton, Adair, McKenzie, Patten, & Perry, 2007; Nutley, Walter, & Davies, 
2007), is informing our understanding of the relationships among research, policy and practice. It 
is clear from this work that research does connect to policy and practice, but that these 
relationships are both multidimensional and multidirectional. There is no universal pattern, 
though it is probably more typical that research first acts on people’s ideas and beliefs, with 
those changes later translating into changes in policy or behaviour. Because all elements in this 
formulation, from ideas to policies to practices, are subject to multiple influences, it is very 
difficult to sort out the effects of research from other influences such as self-interest, political 
forces, or external decisions. 

Our starting point for thinking about the way research connects to education policy and 
practice is a framework developed by Levin (2004) and similar to framing by Nutley et al. 
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(2007) that identifies three key areas related to KM: (1) characteristics of research (such as 
accessibility or perceived quality), (2) characteristics of the educators and schools (such as 
research background, interest level, supporting processes and structures), and (3) the role of third 
parties (such as the media, professional organizations, employers or private providers of 
professional learning) as distributors of knowledge. 

The overall body of scholarship in this field identifies some key conclusions: 

• While it is possible to identify many areas in which research has altered beliefs 
and practices in education (e.g. corporal punishment, special education), it is also 
possible to identify many areas in which quite compelling evidence is not yet 
reflected in common practice (e.g. formative assessment, student engagement). 

• There are many concerns about the quality, relevance and accessibility of research 
in education to practitioners and policy-makers (e.g. small sample sizes, lack of 
methodological rigour, the use of academic language or the publication outlets in 
which research tends to appear). 

• Educators report a high level of receptivity to research but a relatively low level of 
active engagement with research in the sense of spending time reading or 
discussing it. 

• Knowledge mobilization is a function at least as much of social and organizational 
settings as it is of the characteristics of individuals. 

• Education organizations tend to have little organizational capacity to support 
knowledge mobilization, lacking systems, roles and procedures that would make 
research an important part of ongoing activity. 


Barriers to Greater Use of Research 

The barriers to greater research use in schools and other organizations have been well 
described (Bransford et al., 2009; Estabrooks et al., 2003; Mitton et al, 2007; Nutley et al., 2007). 
Some barriers are related to the research itself - for example its inaccessibility, inconsistent 
results, lack of synthesis across multiple studies, or failure to be clear about practice 
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implications. Other barriers are related to the knowledge and skills of individual practitioners, 
who may not know enough about research to be able to find current work, to assess the quality of 
the work, or to understand the meaning in practice of findings expressed in terms of, for 
example, significance levels or effect sizes. 

Many efforts to improve research impact have related to these barriers and have been 
focused on making research more accessible, whether by providing it in forms that are more 
meaningful to practitioners, or by creating more syntheses of research, or by training more 
professionals to understand academic research. 

However these efforts, laudable as they are, are clearly insufficient. A consistent finding 
in the literature is that simply providing information about research findings and implications 
does not change people’s behavior (Bhattacharyya, Reeves, & Zwarenstein, 2009; Nutley et al., 
2007). Knowledge of research findings does not necessarily translate into policy and practice, a 
subject of frustration for many researchers. 

Why is this so? A consistent finding in the literature is the importance of social settings 
and interpersonal relationships in shaping professional practice. Many studies report that 
managers and professionals tend to rely more on their own experience and the views of 
colleagues than they do on the research evidence (Dobbins et al., 2007; Maynard, 2007). This is 
also the conclusion reached by Mitton et al. (2007) based on an extensive review of research in 
health. Cordingley et al. (2004) reviewed evidence on teachers’ adoption of new practices and 
the role of research. They noted that personal recommendations from colleagues affected what 
research was even considered by teachers, let alone whether it was accepted. 
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Organizational Practices Matter 

So, while individual factors such as knowledge of research matter, organizational factors 

matter more. Indeed, our claim would be that there has been too much focus in the relevant 

education literature on the characteristics of individual educators and not enough on knowledge 

mobilization as a feature of organizations and systems. 

The conclusions from empirical research, in both education and nursing, confirm 
that the main barriers to knowledge use in the public sector are not at the level of 
individual resistance but originated in an institutionalised culture that does not 
foster learning. (Hemsley-Brown, 2004, p. 462) 

Coburn and Talbert (2006) studied the use of evidence in school districts and concluded that it 

was greatly affected by structures within the district such as networks, dissemination practices 

and the role of leadership. Individuals’ conceptions of valid evidence, evidence use, and 

research-based practice varied according to the nature of individuals’ role and work. They 

conclude that “organizational structure shapes individual beliefs by influencing patterns of social 

interaction through which they develop” (p. 472) and leaders play a key role in fostering or 

interrupting use of research (p. 491). In the end, organizations shape people rather than each 

person shaping the organization. 

The social nature of belief and behavior presents both an obstacle to and opportunity for 
greater knowledge mobilization. Creating organizational supports and incentives that give 
greater prominence to the consideration of research findings and their implications may shift 
patterns of practice in ways that individualized strategies do not (Cordingley, 2008; Walter et al., 
2003). 

Although organizations can play a critical role in this regard, few service organizations, 
including schools, are set up to do so. Belkhodja, Landry, Amara, and Ouimet (2007) studied the 
“absorptive capacity” of organizations in regard to research. Their survey of a large number of 
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managers in various health care organizations in Canada found quite low levels of research 
absorption in most organizations. They identified specific knowledge integration activities and 
research backgrounds of managers as important influences on research take-up, noting how these 
elements are connected to organizational culture and learning. The same would be true in school 
systems. 


Interventions to Strengthen Knowledge Mobilization 

There is growing literature in the health sector studying interventions to increase KM. 
Interventions are categorized in a number of ways. The most complete taxonomy to date is that 
by Walter, Nutley, and Davis (2003), who categorize interventions based on (a) intervention type 
(some examples from their extensive appendices include research-access, networks, educational 
outreach, provision of expert support, collaboration and so on) and (b) in terms of eight different 
intervention mechanisms (such as education, social influence, incentives, facilitation, or 
reinforcement). In practice, they note, many interventions display aspects of more than one of 
these approaches. 

Assessing the impact of an intervention is difficult because it requires measurement of 
different forms of research use and some way of tracing that use to impact on policy and 
practice. Disentangling the causes of human behaviour or the operation of institutions is 
notoriously difficult to do, but a number of efforts have been made to do so (Lavis et al., 2003; 
Kitson, Harvey, & McCormack, 1998). Kuruvilla et al. (2006) suggest that impacts can be 
thought of as affecting one or more of four categories: research, policy, service, or societal 
broadly. 
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Methodology 

Our approach was to try to change organizational practices in the participating districts 
through several different interventions. If practices shifted, so should the attitudes and 
knowledge of school and district leaders, as one essential element in the mobilization of research 
knowledge. To assess this possibility we used a pre-post design. We first used a survey of 
education leaders in participating school districts to measure organizational practices and 
knowledge around research, followed by interventions intended to change those practices and 
knowledge, followed by another survey to measure whether those interventions resulted in 
changes in knowledge or practice. 

In May 2008, 188 educational leaders in eleven Canadian school districts completed a 
survey regarding their districts’ research practices and their views about six knowledge claims 
related to student success in secondary schools. The districts were located in four provinces and 
varied in size from very small (a few thousand students) to quite large (60,000 students). Urban, 
suburban and rural districts were all part of the study. The survey targeted superintendents, 
secondary school principals and vice-principals, and others holding leadership positions in 
secondary schools, such as department heads. 

We worked with a lead contact in each district who then invited appropriate colleagues in 
the district to take part in the online survey. In total, the districts had about 100 secondary 
schools. We estimated the total population of leaders across the districts at approximately 350 
people. 

The online survey was entirely confidential; neither we nor the district leads knew who 
had taken part. The survey was made up of demographic questions, questions pertaining to 
research use and practice and questions based on research-based propositions (knowledge 
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claims) relevant to secondary school education. In total there were 85 data elements. In an effort 
to reduce positive response bias that is a problem in many studies of research use, our questions 
asked primarily about behaviour rather than about attitudes. 

We asked about organizational practices related to research, such as the extent to which 
research was discussed at various district or school events, their participation in research events 
such as conferences, the extent of use of local data in as annual plans and reports, the degree to 
which their district supported various research activities, and the amount of time spent in the last 
30 days on various research activities. 

To measure knowledge we asked education leaders’ about six knowledge claims about 

secondary education. A knowledge claim (KC) is a finding with strong support from empirical 

research. For example, our first knowledge claim was: 

• Students who fail a single course in the first year of secondary school are at a 
much greater risk of dropping out of school. 

For each knowledge claim we asked about: 

1. whether these leaders’ current beliefs were consistent with available evidence. 

2. the sources of evidence they drew on for their beliefs (including). 

3. the relative importance of research reports, professional development, colleagues 
and professional networks, personal experience, and local data in shaping their 
views. 

In addition to the survey data, we use detailed notes we kept on our interactions with each district 
through the intervention phase, as our contacts discussed with us and colleagues in other districts 
the progress, challenges, and impacts of the study. 
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Findings from Pre-Intervention Survey 

The findings from the first survey are described in detail in Levin, Sa, Cooper, and 
Mascarenhas (2009). To summarize, respondents reported strong interest in research. 85% either 
agreed or strongly agreed that “the important role of research was evident in the ways their 
districts related research to practice”. However when it came to measures of actual practice, such 
as time spent on research related reading, events or network, two thirds to three quarters reported 
quite low levels of involvement. 

Of the 6 knowledge claims, in the initial survey most respondents (at least 79%) agreed 
with the weight of empirical evidence for 3 claims while on 3 other claims, there was much less 
agreement with at least a quarter disagreeing with the evidence. 

In relation to the factors influencing respondents’ views: 

• For all the knowledge claims, respondents report multiple sources of influence on 
their views, but personal experience is the most powerful influence, followed by 
colleagues or professional networks. 

• Direct contact with formal research sources and professional development 
appeared to play a weak role in shaping opinions. 

Respondents reported more use of evidence-based sources, such as research reports and data 

collected in the school, where their beliefs were more consistent with the evidence. There was 

little variation among districts in these responses. 

Creating Interventions to Increase Research Use 

Our goal was to determine if relatively simple interventions, that would be feasible in 
most school districts with current resources, could change these patterns of knowledge and 
behaviour. This approach was based on the suggestion by Lavis et al. (2003) to measure the 
impact of interventions by assessing a change in awareness about a particular body of research 
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knowledge. After sending the data from the survey to the participating districts, we worked with 
them over the 2008/2009 school year to implement one of three interventions to increase 
research use, improve research culture and increase awareness and agreement on the knowledge 
claims. We focused the content of the interventions on the three knowledge claims where there 
was the least agreement with the empirical evidence. At the same time, each intervention would, 
we hoped, have the effect of increasing the amount of attention given to research in the 
participating districts. 

We intended the three interventions to vary from quite passive to much more active, and 
from being primarily focused on sharing information about research to being primarily focused 
on active measures to use research findings. These choices reflect the discussion earlier in the 
paper on current knowledge about the impact of interventions to increase knowledge 
mobilization. However each intervention had to be realistic in the view of the participating 
districts, and we wanted districts, as partners in the work, to have some say in which intervention 
they undertook. 

The three interventions were: 

• Intervention 1—Sharing research articles: The first activity gave districts a 
website with executive summaries and full reports of high quality research on 
secondary schools and student success to be used as each district chose. Nutley et 
al. (2003) suggest increased online dissemination of research products as a 
possible intervention. Other research also suggests that using short research 
summaries, rather than full reports, has the potential to increase use and impact 
(Cordingley, 2008; French, 2005; Landry, Amara, & Lamary, 2001; Nutley et al., 

2007). 

• Intervention 2 —Study groups: The second activity involved creating study groups 
of six to ten district leaders to meet a few times a year to discuss research on 
secondary school improvement. Districts were provided with research reports, 
executive summaries and guiding questions. This intervention arises from 
literature that suggests creating structured time for practitioners to discuss 
research increases impact (Cordingley, 2008; French, 2005; Honig & Coburn, 

2008; Lavis, 2006; Taylor-Powell & Boyd, 2008). 
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• Intervention 3—Districts conducting research: The third activity involved 
districts conducting research to track former students’ post-high school 
destinations, and to use these data to inform district planning for secondary 
schools. Districts were provided with a methodology and survey instrument for 
this activity, which was carried out by secondary students as part of a course. This 
intervention arises from the literature that suggests evidence may be more 
persuasive when stakeholders are involved in a collaborative process to design 
and conduct research initiatives that reflect the local context (Bartunek, Trullen, 
Bonet, & Suaquet, 2003; Cordingley, 2008; Denis & Lomas, 2003; Lavis, 2006; 
Nutley et al., 2003). 


As these interventions proceeded, we tracked their implementation with the participating 
districts through e-mails, conference calls, meetings with participants and, in some cases, visits 
to schools and classrooms participating in the interventions. All of the conference calls and 
meetings were recorded and transcribed. In total we accumulated a large body of data around our 
interactions with the districts. These data were carefully reviewed through normal qualitative 
methods (coding and extracting themes). All authors of this paper were directly involved in these 
interactions, so all authors had direct knowledge of the interventions as well as access to the 
accumulated data. 


Implementing Interventions 

Intervention 1: System to Share Research Articles 

Five school districts were initially assigned this research activity but only three carried 
out the activity to any considerable extent. We created a webpage for each district that was 
tracked using Google Analytics so that we could determine numbers of visitors, downloads of 
resources, and so on. The two school districts that did not attempt to implement the interventions 
also had low response to the initial survey. 
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The three active districts in this intervention included one small and two medium-sized 
school districts. One district selected two reports focusing on student success. They framed the 
reports with guiding questions to look at the application of the research. They found that there 
was a more positive response to the research in this format. Another district, though they 
incorporated the concept of research in the work that they are doing with collegiate renewal by 
using an approach allowing teachers to find relevant research, did not specifically use the 
resources on the website. This is consistent with the literature that suggests more passive 
strategies for dissemination have less impact (Nutley, Jung, & Walter, 2008). 

A third district, however, not only made extensive use of the materials but also contacted 
the research team asking for more! This district used an existing structure, a committee looking 
at improving graduation rates, as a conduit to distribute and discuss the research. Facilitators 
from this committee created a binder for each member which included all of the materials from 
the website. In meetings they began by using the executive summaries. They then asked 
members to focus on two articles and to share the interesting facts in depth with the rest of the 
committee. A third step in the use of the materials was to compare district data to match trends 
that were exposed in the national picture presented in the research reports. 

Intervention 2: Study Groups around Research Issues 

In this intervention, we provided districts with research related materials for three study 
group sessions. Materials included executive summaries, the full report as well as guiding 
discussion questions. We were not involved in organizing these groups or coordinating the 
meetings; districts decided the best way to proceed with these tasks. 
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Three districts, one medium and two larger-sized, implemented this intervention. They 
created study groups or used their existing study group to meet throughout the year to discuss 
research related to student success and student pathways and trajectories. 

One district has had study groups in place since 2003 which were initially led by the 
former superintendent, but today are organized by rotating facilitators. These groups involve 
both secondary and elementary school leaders. A second district had a group of five school 
principals who initiated the group themselves; in this district there wasn’t a facilitator. A third 
district employed their “Coordinator for research and information” at the district level to recruit 
nine principals and to facilitate the sessions. 

Responses from educational leaders participating in this intervention were positive. One 
group struggled in keeping the discussions related only to secondary school students since their 
existing groups consisted of elementary and secondary leaders, and they did not use the study 
materials provided. Another group had difficulties in meeting regularly, because they did not 
have anyone facilitating the group. The third district was the most successful in implementing 
this intervention, which can be attributed to focused groups, and having a facilitator who 
organized the group and ensured engagement with the materials by the principals. 

Some impacts did occur from these groups. In one district the group presented research 
related to secondary school student success at a PD session for the Association of Secondary 
School Administrators for the district. This study group also invited a speaker from an 
organization that evaluates programs that support students to go on to post-secondary education, 
broadening their network with external organizations. Districts leaders from the three districts 
that were introduced to each other through the research team’s telephone conferences have been 
communicating with each other through email sharing models for school improvement. 
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Intervention 3: Districts Conducting Research 

The research team provided districts with resources to conduct a post-school destination 
survey of previous students. This included setting up the survey online and sending the districts 
the data files after they had administered the survey. The research team also provided each 
district with an instructional package outlining how to conduct data analysis. The research team 
was not involved in co-coordinating and planning different avenues to present the findings of the 
post-school destination surveys to leaders; participants decided the best way to share their 
findings with district leaders. 

Two small districts and one large district attempted to implement this intervention. In the 
two small districts the project was carried out through central office by the district vice principal 
or superintendent. The large district was the only district participating in the study that had a 
formal research infrastructure in the form of a district research team. This team facilitated the 
project. All three districts had experience in conducting similar surveys of graduates in the past 
through the central offices. 

Each district required different resources and support from the research team throughout 
this intervention. One district used only the initial support, and another modified the survey 
instrument somewhat. Though the district leaders made efforts to combine the “Post School 
Destinations” surveys with other surveys being administered in the district, they lacked the time 
and the districts lacked the capacity to work with teachers and students to collect and analyze the 
data. The intervention in these two districts did not move beyond creating and uploading the 
survey. 

The third district had substantial ongoing support to implement this project and also 
seemed to have the most success with the intervention. The district was able to carry out the 
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intervention to completion in three secondary schools, and the intervention was carried out as 
part of a math course. At the end of the term, students presented their findings to their school 
vice principal, math teacher and representatives from the district research team. 

The district research team and district leaders in the other two participating districts 
identified the need to align the various surveys that are being carried out in their districts. One 
student in the large district participated in an internship in which she worked with three surveys 
that were conducted in her school that year around student engagement and how schools prepare 
students for post-school life. She created a report and video which shared and compared results 
from the three surveys demonstrating that the various initiatives can be aligned and the data can 
be used to provide a more comprehensive view of student experiences. 

Overall, these interventions had modest success. Although nine districts agreed to take 
part in the interventions, only three districts were able to move the interventions to any 
significant level of action. The usual barriers to action emerged, including lack of time, lack of 
capacity, the absence of someone to take initiative, and the relatively low priority these activities 
received. We return to this issue at the end of the paper. 

Post-Intervention Survey 

Following the intervention phase of the study, in the fall of 2009, secondary school and 
district leaders were invited to respond to the same survey as in May 2008. 158 administrators 
completed this second administration of the original survey by January 2010. The data reported 
are highlights of observations in comparing the pre and post intervention survey data. These data 
must be treated very cautiously for two reasons. First, the interventions were not effectively 
implemented in many cases. Secondly, due to anonymity provisions, we do not know how many 
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of the same people completed the pre and post surveys; there are grounds for thinking that the 
populations for the two surveys are significantly different. 


Overall Research Practices 

In most cases there were no significant shifts in responses about research practices from the pre 
to the post survey. There were five questions (out of 39) where we found noticeable shifts in the 
survey data. However we do not have grounds for attributing these changes to the interventions. 


Table 1 

Summary of Overall Research Practices 


Survey Question 

Item (where applicable) 

Differences in pre-post survey data 

5. How often would you 
say research is discussed 
in your district during the 
following: 

Principals Meetings 

Percentage increase in respondents who reported 
they discussed research often/always from 45% to 

54% 

Administrative Meetings 

Often /Always from 42% to 57% 

7. This school district 
follows these practices: 

Circulates research articles 

Increase from 74% to 89% Yes in circulating 
researching articles 

Incorporates/links data to 
reporting 

Increase from 78% to 88% Yes 

8. Does your school 
district regularly report 
and analyze the following 
data sources: 

Special education referral or 
placement rates 

Increase in Yes from 60% to 67% 

9. Are local data or other 

research cited within the 
following district and/or 
school documents: 

District annual report 

Increased reporting from Yes of 76% to 87% 

School Improvement Plans 

Increased reporting from Yes 83% to 87% 

10. How often does your 
school district offer the 
following research 
activities/strategies: 

Research related resources 

Increase in monthly use from 43% to 53% with a 
(4% net increase) 


These data are not strong enough to permit any conclusions across the group as a whole about 
the impact of the interventions. 
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Individual District Research Practice 

We ran the pre-post analysis also for each district. Only six districts had enough 
responses both pre and post to permit a reasonable analysis. The increases in research practices 
are small and inconsistent across districts. Given these findings and concerns about 
comparability of respondents, we are unable to draw any reasonable conclusions about the 
impact of these interventions affected research practice in the districts. 

Overall Data for Knowledge Claims 

Similarly, as shown in Table 2, there was relatively little change across the two surveys in 
responses to the six knowledge claims, with no consistent pattern across the claims either in 
agreement or in the use of research to shape opinion. The table also shows the 5 of respondents 
who identified research and data as important or very important in both the pre and post surveys. 
Again, there is no systematic change in these responses. 


Table 2 

Overall agreement with knowledge claims in conjunction with importance of data and research 
as sources of knowledge 



KC1 (A/SA) 

% 

KC2 (A/SA) 

% 

KC3 (A/SA) 

0/ 

KC4 (A/SA) 

% 

KC5(D/SD) 

% 

KC6(D/SD) 

% 


Pre Post 

Pre Post 

/l 

Pre 

1 

Post 

Pre Post 

/ o 

Pre Post 

/o 

Pre Post 

Agreement 

with claim 

63 

82 

93 

96 

79 

82 

87 

85 

39 

34 

37 

33 

Importance 

of Research 

36 

57 

53 

61 

46 

45 

50 

44 

34 

39 

28 

33 

Importance 

of Data 

55 

58 

57 

56 

48 

48 

44 

43 

36 

30 

41 

40 
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However it is interesting to note that where increases in agreement with the research evidence 
occurred, research and data were reported as more important sources of knowledge. 

Individual District Data for Knowledge Claims 

For each of the three interventions there was at least one district that was more successful. 
However the districts that reported and were observed to have high levels of engagement with 
the interventions did not have more change in the post survey than other districts using the same 
intervention. One possible reason for this is that these interventions did not necessarily have a 
wide spread across districts. For instance, the study groups in total included on average 8-10 
principals per group, per district. So the materials were only utilized by a small proportion of 
respondents. Moreover, for reasons mentioned earlier, we cannot be sure that the administrators 
that participated in the interventions participated in the post survey. 

The findings from this analysis are consistent with the literature on interventions that says 
that “success” varies widely across practice contexts. It is difficult to implement interventions to 
increase research use, and even when implemented, interventions may have limited impact. 

Factors Supporting the Interventions 

While we cannot trace effects from these interventions on research practices or 
knowledge, our qualitative data led us to identify four factors that made implementation of the 
interventions stronger. These findings have relevance for future attempts to design and 
implement measures to increase research use in school districts. 

• Formalized Organizational Structure 

• Making research accessible and relevant 
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• Alignment and link to practice 

• Facilitation 

Formalized Organizational Structure 

Research use is likely to be stronger where it is supported by organizational structures. 
Our finding here is consistent with others, such as Cobum, Honig, and Stein (2009). Effective 
research use is not just a matter of individual skill or volition. It requires some kind of process in 
the school and/or district for educators to engage with research, such as a structure, an agenda, or 
some venue where research gets talked about. In most of the districts formal structures and 
processes for research use were lacking, and where districts did have them, they were often 
undemtilized. For instance, one district had a central portal to share research related resources, 
but use of this portal was minimal. 

Districts need systems to find and share research. Participants spoke of research use 
being a result of multiple factors. At the very least, availability of material is the baseline for 
research use. Only one district reported having a common area where educators can find and 
share relevant research, and it is unclear whether or not most educators in the district know about 
it or used it. The creation of websites or chat rooms is a popular strategy, but our evidence and 
related literature suggest that in many cases such efforts have little or no effect (Edelstein, Shah, 
& Levin, 2011). 

Districts need to incorporate research use into formal structures and processes. 
Educational leaders spoke of the need for structures and formalized processes that increased the 
likelihood that educators would engage with research. It should come as no surprise that 
educators are busy and face a number of complex challenges daily. Educators often spend their 
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days reacting to unexpected events rather than enacting planned efforts to increase research use. 
Currently, most use of research occurs in the form of ad-hoc conversations. 

Structures are not necessarily enough either, though. It cannot be assumed that building 
the right structures equals successful KM. Even with research capacity, there may be a culture in 
the district that inhibits KM - for example a hierarchical organization in which disagreement 
with superiors is discouraged. It is important, then, not only to build structures that support 
research use but also to foster a culture that supports and encourages research use, which means 
valuing the open exchange of ideas and the questioning of conventional wisdom. 

Making Research Accessible and Relevant 

One of the most frequently noted features of KM in the literature has to do with the form 
in which research reaches practitioners. Our evidence also suggests that the nature and format of 
research material affects use. Our respondents expressed a strong preference for reports and 
documents that were readable and practical. In the resource website and study group 
interventions (both of which provided educational leaders directly with research) tailoring and 
adapting the research products to the needs of leaders was mentioned as increasing the tendency 
to use it. 

An interesting response from some of our partners that is not much noted in the literature 
was the desirability of research that was provocative and engaging. More than one district leader 
noted that research had more impact where it provoked debate and controversy, or where the 
issues were seen as urgent. 

We also found that guiding questions can be effective in promoting discussion. They can 
act as a reflective lens, increasing the relevance of research. For the second intervention (study 
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groups), we created questions for districts to consider and included these at the end of the 
executive summaries we created for the study groups. Educational leaders told us that the 
executive summaries and the guided questions acted as a reflective lens allowing them to 
consider their own district in relation to the broader provincial and national picture. 


Alignment and Linking to Practice 

Also often noted in the literature is the importance of potential users of research seeing 

the connection to their own work and immediate interests. In this study, the interventions worked 

best in districts that made efforts to connect this project to their existing structures and priorities. 

Our respondents made this point repeatedly. 

One aspect of connection was the match with topics that were already seen as important 

or already under discussion in the school. Several of our partners noted that the interventions 

worked where they were connected with work already being done. 

We have a committee in our district looking specifically at non-completion. We 
have representatives from elementary, middle and secondary and our mid-teams 
as well. We created a binder for each participant on the team. In most cases, we 
just used the executive summary to start. Then, we tried to find whether we had 
district data to match the trends that were exposed in the national picture. Out of 
this committee, we are now looking at action. We keyed in on the link between 
attendance and students dropping out of school. Now we have a subcommittee 
looking at attendance and tracking right back into elementary. (Lead Teacher, 

Website District B) 

Another source of relevance comes from alignment with other initiatives either in the district or 
at a provincial level. Where other parts of the organization or other levels of the education 
system are asking similar questions or raising similar issues, there is heightened interest in 
research: 

We have a Ministry Satisfaction survey in our province, so we put our grad survey 
this year into the Ministry Satisfaction survey. We just have over surveying going 
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on in our district... A lot of teachers are not seeing the purpose of so many 
surveys and are asking what are we doing, what is done with the results that we 
are getting. (District Vice Principal, Data District A) 


Facilitation 

Many educational leaders spoke to the need for a facilitator to increase research use in 

schools. They articulated a variety of roles that could help including coordinating and setting up 

meetings, picking research materials and preparing executive summaries and guiding questions 

as well as facilitating the research discussions to keep it focused and make it more meaningful. 

This view is consistent with the literature on the importance of intermediaries, and also supports 

the importance of structures, processes and cultures of research use: 

We are looking at ways of supporting teachers to access and utilize 
research... .hence... .the need for a facilitator, and I think for many of us, that’s 
still where we are at. Increasing research use still requires that. For the most part, 
the majority of teachers will not seek research out on their own, unless there is 
someone there that prompts them to do so or facilitates it. (Coordinator, Website 
District A) 

I think one of our struggles is that we don’t have a fonnal facilitator. (Secondary 
School Principal, Study Group District B) 

In the districts where the interventions gained most purchase, much of the credit went to 
individuals who played this facilitative role. Those individuals had varying titles and 
responsibilities, ranging from superintendents to district principals to research officers, but they 
were all connected by the desire to increase the profile of research in the district, and by their 
ability to bring people together to that end. 
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Conclusion 

Our experience in trying to generate improvements in the research practices of districts 
and the research knowledge of district leaders based on some fairly simple interventions is 
mixed. Even these modest interventions were not adopted in many districts, suggesting that there 
is either no appetite or no capacity (or both) to increase research use, despite the initial survey 
results which were enthusiastic about the potential value of research. Here our results are 
consistent with other research that shows how difficult it is to change organizational behaviour 
even when there is willingness to do so, both because the interventions are hard to carry out and 
because even when carried out they may have limited impact on what people do. Our findings 
also support previous research showing that the provision of research information is an 
insufficient approach to changing practice. At the same time, both in terms of research practices 
and knowledge of key research claims, there are grounds for optimism, as significant levels of 
interest in a stronger role for research were reported in our survey. 

Where the interventions had the most impact, it was largely due to the presence of 
advocates and intermediaries or facilitators in the districts. In each of these cases a person or 
persons (usually one key person) led the way in supporting the intervention and in championing 
the cause of increased use of research. 

Our work also points to the need to develop better methods for studying knowledge 
mobilization. We do not need any more studies of perceived barriers to research use. Also, 
researchers should be trying to reduce the reliance on surveys of attitudes in favour of more 
attention to knowledge or behaviour. However these shifts will require more sophisticated 
research approaches, and as indicated in this study, these pose some complex methodological 
challenges. 
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Overall, these results suggest that there is much still to learn about the kinds of steps or 
actions that are most likely to result in stronger knowledge mobilization in schools and school 
districts. 
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